Skip to content

Retry waiting for javac under Vulpix #17373

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 4, 2023
Merged

Retry waiting for javac under Vulpix #17373

merged 1 commit into from
May 4, 2023

Conversation

som-snytt
Copy link
Contributor

If running a single test, the pool shutdown
is likely to subvert waitFor the javac process.
Catch the InterruptedException and try again
with a timed wait of generous but finite duration,
to accommodate testing by developers. This quick
fix does not correct the race, which presumably
does not matter because of the order in which tests
are ordinarily submitted.

For example, tests/pos-java-interop/t2940:

sbt:scala3> testCompilation t2940
[snip]
[info] Test dotty.tools.dotc.CompilationTests.posTwice started
[                                        ] completed (0/1, 0 failed, 2s)java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1640)compiler / Test / testOnly 3s
        at java.base/java.lang.ProcessImpl.waitFor(ProcessImpl.java:447)
[snip]
java.util.concurrent.ExecutionException: java.lang.InterruptedException
        at java.base/java.util.concurrent.ForkJoinTask.reportExecutionException(ForkJoinTask.java:581)
        at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:993)
        at dotty.tools.vulpix.ParallelTesting.dotty$tools$vulpix$ParallelTesting$Test$$_$executeTestSuite$$anonfun$4(ParallelTesting.scala:687)
[snip long stack trace]

@som-snytt som-snytt marked this pull request as ready for review April 29, 2023 23:33
@anatoliykmetyuk anatoliykmetyuk requested a review from dwijnand May 1, 2023 13:47
@som-snytt
Retry waiting for javac
0975d29 
If running a single test, the pool shutdown
is likely to subvert `waitFor` the javac process.
Catch the `InterruptedException` and try again
with a timed wait of generous but finite duration,
to accommodate testing by developers. This quick
fix does not correct the race, which presumably
does not matter because of the order in which tests
are ordinarily submitted.
@som-snytt som-snytt force-pushed the tweak/vulpix-spurious-interrupt branch from 05aa3ab to 0975d29 Compare May 3, 2023 22:25
@dwijnand dwijnand merged commit 983a314 into scala:main May 4, 2023
@som-snytt som-snytt deleted the tweak/vulpix-spurious-interrupt branch May 4, 2023 15:11
@som-snytt
Copy link
Contributor Author

Locally, I just saw a couple of interrupted tests while scala3-bootstrapped/test.

[info] Test dotty.tools.dotc.BootstrappedOnlyCompilationTests.posMacros started
[======================================> ] completed (327/328, 0 failed, 21s)java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1722)
        at java.base/java.lang.ProcessImpl.waitFor(ProcessImpl.java:447)
        at dotty.tools.vulpix.ParallelTesting$Test.compileWithOtherCompiler(ParallelTesting.scala:628)
        at dotty.tools.vulpix.ParallelTesting$CompilationLogic.compileTestSource$$anonfun$1$$anonfun$1(ParallelTesting.scala:238)
        at scala.collection.immutable.List.map(List.scala:246)
        at dotty.tools.vulpix.ParallelTesting$CompilationLogic.compileTestSource$$anonfun$1(ParallelTesting.scala:238)
        at scala.util.Try$.apply(Try.scala:210)
        at dotty.tools.vulpix.ParallelTesting$CompilationLogic.dotty$tools$vulpix$ParallelTesting$CompilationLogic$$compileTestSource(ParallelTesting.scala:239)
        at dotty.tools.vulpix.ParallelTesting$$anon$4.checkTestSource$$anonfun$1(ParallelTesting.scala:285)
        at dotty.tools.vulpix.ParallelTesting$$anon$4.checkTestSource$$anonfun$adapted$1(ParallelTesting.scala:288)
        at scala.Function0.apply$mcV$sp(Function0.scala:42)
        at dotty.tools.vulpix.ParallelTesting$Test.tryCompile(ParallelTesting.scala:462)
        at dotty.tools.vulpix.ParallelTesting$$anon$4.checkTestSource(ParallelTesting.scala:288)
        at dotty.tools.vulpix.ParallelTesting$Test$LoggedRunnable.run(ParallelTesting.scala:358)
        at dotty.tools.vulpix.ParallelTesting$Test$LoggedRunnable.run$(ParallelTesting.scala:340)
        at dotty.tools.vulpix.ParallelTesting$$anon$4.run(ParallelTesting.scala:283)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1403)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
Interrupted (probably running after shutdown)
java.util.concurrent.ExecutionException: java.lang.InterruptedException

and

[info] Test dotty.tools.dotc.CompilationTests.explicitNullsRun started
[===============================>        ] completed (8/10, 0 failed, 0s)java.nio.channels.ClosedByInterruptException
        at java.base/java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:199)
        at java.base/sun.nio.ch.FileChannelImpl.endBlocking(FileChannelImpl.java:171)
        at java.base/sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:430)
        at java.base/java.nio.file.Files.createFileChannelLinesStream(Files.java:4123)
        at java.base/java.nio.file.Files.lines(Files.java:4109)
        at dotty.tools.utils$package$.toolArgsFor$$anonfun$1(utils.scala:68)
        at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183)
        at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179)
        at scala.collection.immutable.List.foldLeft(List.scala:79)
        at dotty.tools.utils$package$.toolArgsFor(utils.scala:74)
        at dotty.tools.vulpix.ParallelTesting$TestSource.allToolArgs(ParallelTesting.scala:103)

@yilinwei
Copy link
Contributor

yilinwei commented Feb 7, 2024 •
edited
Loading

According to the javadoc for the ExecutorService, this shouldn't happen.

The doc for shutdown states that:

Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.

Since we call awaitTermination straight afterwards with a long timeout, all the tasks ought to be finished.

@som-snytt som-snytt mentioned this pull request Oct 17, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dwijnand dwijnand dwijnand approved these changes

Assignees

@dwijnand dwijnand

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@som-snytt @yilinwei @dwijnand